New kernels for fast mesh-based motion estimation

Mesh-based motion estimation—also known as control grid interpolation or warping—provides a smoother estimated intensity field compared to the traditional block-matching algorithm (BMA), resulting in most cases in a more realistic motion field and smaller estimation error. In mesh-based motion, unlike BMA, the computation of a motion vector is affected by its neighboring vectors. This interdependence necessitates a costly, iterative computation of motion vectors. The computational cost of meshbased motion has been a main drawback of this otherwise powerful technique. We propose to use noniteratively computed motion vectors, such as BMA motion vectors, for node motions in the mesh model. However, we found that a straightforward insertion of BMA motion vectors in the deformable mesh leads to unpredictable and erratic results, and were thus motivated to carefully analyze the interaction of motion vectors and interpolation kernels in mesh models. This analysis leads to a methodology for computing optimal motion interpolation kernels for a given set of motion vectors (e.g., BMA motion vectors). We find a generalized orthogonality condition for these kernels; optimality is achieved only if the projections of vertex motions on the local intensity gradients are statistically orthogonal to mesh-based estimation errors. Experiments show that optimal kernels are often very different from the traditional bilinear kernels, and exhibit interesting variations. The new kernels benefit a variety of applications, including motion estimated interpolation, denoising, and compression.